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ABSTRACT 

The applicability of the Analysis of Variance, ANOVA, 
procedures to the analysis of dichotomous repeated measure data is 
described. The design models for which data were simulated in this 
investigation were chosen to represent simple cases of two 
experimental situations: situation one, in which subjects* responses 
to a single randomly selected set of three stimuli or items were 
evaluated dichotomously on four successive occasions, and situation 
two, in which subjects* responses are evaluated dichotomously on four 
successive occasions with respect to four separate randomly selected 
set of three stimuli or items. The two experimental situations are 
represented by "three way factorial 11 designs with random factors for 
subjects and items and a fixed factor for occasions; the first with 
subjects, items and occasions completely crossed and the second with 
items and occasions crossed with subjects, but with items nested 
within occasions. Results of the study are presented in 1 1 tables and 
7 figures. (Author/DB) 
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The problem with which this paper will be concerned is the applicability 
of the Analysis of Variance, ANOVA, procedures to the analysis of dichotomous 
repeated measure data. Recall, that in the employment of ANOVA procedures it 
is assumed that the data can be reasonably modeled by a vector variate with 
multi-variate normal density and diagonal or patterned "compound symmetric" 
covariance matrix, that is with equal variances and equal covariances in 
diagonal blocks for subjects and ;:eros elsewhere, (Greenhouse and Geisser, 

1959). Can these familiar procedures still be. employed when the data can 
more reasonably be modeled by a vector variate with non-independent Bernoulli 
variates as elements? Other investigators, Mandeville (1970) and Seegar and 
Gabrielson (1968) have employed Monte Carlo methods in order to achieve a 
partial answer to the above question. Their results tend toward a cautious 
but affirmative answer with respect to the design models and model parameter 
configurations with which their simulations were concerned. The results re- 
ported in this paper are less sanguine. That is, however, not to say that 
the results reported in this paper are in disagreement with those of previous 
investigators, since the design models and model parameter configurations 
with which this paper is concerned are somewhat different. Bradley (1968) 
has indicated that the robustness of a parametric statistical test is idio- 
syncratic rather than general. For example, the F-test has bean shown to be 

# 

robust to heteroscedasiticity for balanced designs (equal cell n # s), but 
generally not robust to heteroscedasiticity for unbalanced designs. 

The design models for which data were simulated in this investigation 
were chosen to represent simple cases of two experimental situations, situation 
one, in which subjects responses to a single randomly selected set of three 
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stimuli or items were evaluated dichotomous ly on four successive occasions 
and situation two, in which subjects 1 responses are evaluated dichotomously 
on four successive occasions with respect to four separate randomly selected 
sets of three stimuli or items. Table 1 contrasts these two experimental 
situations. 

Table 1. Contrast of experimental Situation 1 with Experimental Situation 2. 



Situation 1 


Occasion 
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I 1 I 2 I 3 
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Situation 2 


Occasion 
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I 4 I 5 I 6 
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In terms of experimental design the two experimental situations can be represented 
by "three way factorial" designs with random factors for subjects and items 1 
and a fixed factor for occasions; the first with subjects, items and occasions 
completely crossed and the second with items and .occasions crossed with subjects, 
but with items nested within occasions. 



Sterns were considered to represent a random source of variation, since 
experimenters often wish to generalize their results to some domain of 
items or stimuli, rather than to restrict their findings to only those 
items they actually employed. 
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These two experimental situations were considered to be of interest because 
experimenters often form occasion measures for subjects as the sum of dichotomous 
response evaluations to a series of items and then ignore items as a factor in 
their experimental design. And because it may be shown, (Draper, 1971), that 
for such experimental situations, there is a source of variation associated 
with items 2 , which if nonrnull will be confounded with the source of variation 
for occasions. A situation which holds whether or not , items are included as 
a factor in the experimental design and an ANOVA consistent with the design 
is employed for analysis and which also holds whether the dependent measures, 
or item scores, are dichotomous or have distributions which may be reasonably 
approximated by a normal probability density. 

For situation one an examination of expected mean squares, suggested 
possible variance ratio tests for sources of variation due to subjects, occasions, 

/ 

items, subjects by occasions interaction, and items by occasions interaction. 

Further, since the items by occasions interaction could be non— null and would , 

i 

thus be confounded with occasions, a Quasi-F variance ratio test for occasions 
(Satterthwaite, 1941) suggested itself. Similarly for situation two possible 
variance ratio tests were suggested for the sources of variation, subjects, 
occasions, items, and subjects by occasions interaction, where both a ’'regular' 1 
variance ratio test and a "Quasi-F" test were suggested to test for occasions 
effects. Calculations with respect to all the suggested tests were made on 
simulated data, but of major interest were those tests for occasions and the 
subjects by occasions interaction. 



2 An item by occasion interaction effect in situation one or a main effect 
due to items in situation two. 
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It was considered of interest to vary the following model parameters over 
examples of the experimental situations to be investigated. 

1) The base probability of one 3 , on item difficulty, in the data, 

2) The number of subjects* 

3) The degree of relative heterogenity of the subjects 4 , 

4) The effect of items 4 , 

5) The effect of occasions 4 , 

and 

6) The effect of subject by occasion interaction 11 . 

Three different values of the base probability of a one, .5, .2, and ,1, suggested 
by results due to Lunney (1969) , were investigated as were five values of the 
number of subjects 4, 6, 8, 10, and 12. The four levels of relative subject 
heterogenity. Levels 1, 2, 3, and 4, which were employed, correspond to four y 

ratios of subject to error variance, 2:8, 3:7, 4:6 and 5:5 respectively. Com- 
binations of these parameter values in combination with the values for effects 
due items, occasions and subject by occasion interaction resulted in 720 different 
model parameter configurations for which data were simulated. The data for 
the model parameter configurations were simulated as pseudo-random numbers 
with distributions which could reasonably be approximated by a normal proba- 
bility density function and all ANOVA statistics calculated, then the data 
were dichotomized and the ANOVA calculations repeated. Determinations were 
then made as to which statistics fell to various a rejection regions and counters 
which had been initialized at zero were incremented by one where appropriate. 



3 The base probability of a one will refer to the general probability of a 
one in the absence of possible main and interaction effects. 

^Whether null or non-null. 
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